Image compression processing device, image compression processing method, and image compression processing program

ABSTRACT

A high-frequency integrating circuit  32  detects characteristics of horizontal high-frequency components and vertical high-frequency components of the image formed by a processing-target image signal. Based on the detection result, a CPU  61  obtains the number of bits of the after-compression-coding data of the image signal, and calculates the compression rate dependent upon the number of bits. The CPU  61  controls an image codec  36  so that the processing-target image signal is compression-coded through only one time of compression coding processing by use of the calculated compression rate. This configuration allows the image compression processing (compression coding) to be rapidly executed with high accuracy.

TECHNICAL FIELD

The present invention relates to an image compression processing device,an image compression processing method and an image compressionprocessing program that are used in processing of still image data andmoving image data in various apparatuses such as digital still cameras,digital video cameras, and camera-equipped cell phone terminals.

BACKGROUND ART

Digital video cameras for mainly capturing moving images and digitalstill cameras for mainly capturing still images are widely used. Some ofthese cameras can capture both moving images and still images.Furthermore, so-called camera-equipped cell phone terminals,camera-equipped portable electronic notebooks and so on provided with adigital camera function are also becoming widely used.

In these apparatuses having an imaging function, such as the digitalcameras and camera-equipped cell phone terminals, if moving image dataand still image data obtained through imaging are recorded as they arein a recording medium (storage medium) of which recording capacity isfinite, the recording capacity of the recording medium is fully occupiedsoon since the data amount of these obtained image data is large.

Therefore, when image data arising from imaging is recorded in a storagemedium, the image data is subjected to data compression processing basedon any of various systems so that the data amount thereof is reduced,followed by being recorded in the storage medium. For example, whenimage data to be recorded is moving image data, a data compressionsystem such as the MPEG (Moving Picture Experts Group) system is used.When image data to be recorded is still image data, a data compressionsystem such as the JPEG (Joint Photographic Experts Group) system isused.

A description will be made on one example of an existing imagecompression device. FIG. 9 shows one example of an existing imagecompression processing device that subjects still image data to datacompression by the JPEG system for example. In this existing imagecompression processing device shown in FIG. 9, compression-target imagedata supplied to this device is provided to a DCT (Discrete CosineTransform) unit 101 and is subjected to the discrete cosine transformtherein to thereby be transformed from components along the time axisinto components along the frequency axis. The image data transformedinto the frequency-axis components is supplied to a quantizer 102.

The quantizer 102 adjusts the compression rate of the image data fromthe DCT unit 101 based on quantization table information obtained from afixed-length quantization table creator 107, and supplies the adjustedimage data to a variable-length coder 103. The variable-length coder 103executes variable-length coding for the image data from the quantizer102 by use of variable-length codes such as the Huffman codes. Thevariable-length coder 103 outputs the coded data as compressed imagedata and supplies it to a byte calculator 104.

The byte calculator 104 calculates the number of bytes of the compressedimage data corresponding to one screen based on the coded image datafrom the variable-length coder 103, and supplies the calculation resultto a quantization scale calculator 105. The quantization scalecalculator 105 calculates the difference between the number of bytescalculated by the byte calculator 105 and a predetermined number ofbytes to thereby calculate a compression rate adjustment amount, i.e.,quantization scales. The calculation result by the quantization scalecalculator 105 is supplied to the fixed-length quantization tablecreator 106.

The fixed-length quantization table creator 106 creates a newquantization table based on the newly calculated quantization scales asthe calculation result from the quantization scale calculator 105 and aquantization table 107 supplied from a quantization table unit 107, andsupplies the new quantization table to the quantizer 102. Theabove-described loop processing is repeated multiple times so that theimage data is stepwise compressed into data having a predetermined datasize.

However, because compression processing is executed multiple times(retry of compression processing is repeated) until an adequatecompression rate is obtained as described above, it takes a long periodto complete the compression processing. Therefore, an imaging deviceemploying an image compression processing device like that describedwith FIG. 9 involves a problem that a short imaging interval cannot beused because of the long period for completion of compression processingfor captured image data to be recorded. Furthermore, repeatingcompression processing multiple times needs provision of a high-capacitymemory for holding the entire data of an original image.

In contrast, if compression into a predetermined data size through onlyone time of compression processing is intended, a high compression rateneeds to be used. However, although a data size can be easily decreasedto smaller than a predetermined size, an unnecessarily high compressionrate possibly causes so-called block noise and mosquito noise, whichleads to deterioration of the quality of a reproduced image.

As a solution to these problems, a technique to allow rapid andappropriate data compression through one time of data compressionprocessing is disclosed in Japanese Patent Laid-open No. 2003-199019. Inthis technique, high-frequency components of an image to be recorded areextracted based on a thumbnail image of the image to be recorded, andthe image data amount corresponding to one screen is predicted based onthe extracted components, so that an adequate compression rate is setbased on the prediction result.

DISCLOSURE OF INVENTION

According to the above-described technique disclosed in Japanese PatentLaid-open No. 2003-199019, image data can be compressed more rapidlythan by the existing image compression processing described with FIG. 9.Furthermore, a compression rate obtained in advance is not unnecessarilyhigh, which can avoid image deterioration.

However, in the technique disclosed in Japanese Patent Laid-open No.2003-199019, in order to comprehend the characteristic of an image to becompressed, a high-frequency component integrating means extractshigh-frequency components of the horizontal and vertical directions byuse of a thumbnail image (preparatory image) of which size is greatlysmaller than that of the original image. Therefore, there would be acase where an error is involved in the code amount prediction because ofdeficiency in the image information.

Furthermore, if the code amount corresponding to the entire screen ispredicted only from horizontal high-frequency components like in thetechnique disclosed in Japanese Patent Laid-open No. 2003-199019, thefollowing problem would arise. Specifically, when an input image Ghaving a lateral streak pattern like one shown in FIG. 10 is processedfor example, although vertical high-frequency components are large inthis input image G, this characteristic is difficult to extract and theafter-compression code amount is predicted to be smaller than the actualvalue because the code amount is predicted with a focus only on the bandof the horizontal direction.

If extraction of vertical high-frequency components is also intended inorder to avoid this problem, it is necessary to construct ahigh-frequency component extraction circuit with use of a memory (linememory) for storing therein data of a compression-target imagecorresponding to one line or several lines, which possibly leads to anincrease in the circuit scale and cost up.

Furthermore, in the technique disclosed in Japanese Patent Laid-open No.2003-199019, high-frequency component integration processing and codeamount prediction processing are executed in an image monitoring modefor confirming whether or not an imaging target is being captured. Incontrast, the captured image itself is loaded after transition to a datarecording mode. Therefore, the image subjected to the high-frequencycomponent integration processing and code amount prediction processingin the image monitoring mode does not correspond with the image loadedto be recorded. In imaging with use of a flash in particular, this pointwould cause deterioration of accuracy of the code amount prediction.

Accordingly, image compression processing devices having a function tocompress images conventionally involve the following problems (1) to(7). Specifically, the existing devices involve the problems of (1) along compression processing period due to multiple times of feedbackprocessing of code amount prediction for an image to be compressed, (2)a large memory scale due to the necessity for the entire original imagedata to be stored in order to implement multiple times of feedbackprocessing of code amount prediction for an image to be compressed, and(3) image quality deterioration due to insufficiency of accuracy of codeamount prediction when a compression-target image is compressed throughonly one time of compression processing.

In addition, the existing devices further involve the problems of (4)the occurrence of a code amount prediction error due to extraction ofhigh-frequency components from an image, such as a thumbnail(preparatory image), of which size is greatly smaller than the originalimage, (5) the occurrence of a prediction error in the case of an imageinvolving imbalance of the frequency band between the horizontal andvertical directions when a code amount is predicted only from either oneof horizontal and vertical high-frequency components, (6) a large memoryscale due to the necessity for data corresponding to one line or severallines to be stored in order to extract vertical high-frequencycomponents, and (7) the occurrence of a code amount prediction error dueto disagreement between the compression-target image to be recorded andthe image used for code amount prediction.

In consideration of the above-described problems, an object of thepresent invention is to provide an image compression processing device,an image compression processing method and an image compressionprocessing program that each can clear up the above-described problemsand each have the following features. Specifically, the device, methodand program can execute compression processing (compression coding) forimages rapidly with high accuracy, and can record yet-to-be compressedimage data in a storage medium rapidly. In addition, the device, methodand program allow efficient use of a storage medium in which yet-to-becompressed image data are recorded, and can improve the response ofimage compression processing.

In order to solve the above-described problems, an image compressionprocessing device according to the invention set forth in claim 1includes:

recording control means that is supplied with an image signal andrecords the image signal in a storage medium;

first detection means that is supplied with the image signal and detectscharacteristics of a horizontal high-frequency component and a verticalhigh-frequency component of the image signal;

calculation means that calculates a compression rate in compressioncoding of the image signal by a predetermined coding system based on adetection result from the first detection means; and

coding means that executes compression coding for the image signal or animage signal read out from the storage medium based on the compressionrate calculated by the calculation means.

According to the image compression processing device of the inventionset forth in claim 1, the first detection means detects characteristicsof horizontal and vertical high-frequency components of the image formedby an image signal to be processed. Based on the detection result, thecalculation means calculates the compression rate of the image signal.Furthermore, in accordance with the compression rate from thecalculation means, the compression coding means can compression-code theimage signal into image data having a desired data amount through onlyone time of compression coding processing.

Due to this configuration, the compression coding processing can beexecuted rapidly, and whether the image is one involving drastic changesor one involving small changes can be determined by use of thehorizontal and vertical high-frequency components. Thus, the compressionrate can be calculated accurately without an error. Accordingly, alsowhen a compression-coded image is subjected to expansion processing soas to be restored, the occurrence of image deterioration is avoided. Inaddition, because the image compression processing can be completedthrough one time of image coding processing, if yet-to-becompression-coded image data is stored and held in the storage mediumfor example, the storage medium can be utilized efficiently.

An image compression processing device according to the invention setforth in claim 2 is dependent upon the image compression processingdevice set forth in claim 1, and further includes second detection meansthat is supplied with the image signal and detects characteristics of ahorizontal low-frequency component and a vertical low-frequencycomponent of the image signal. Furthermore, the calculation meanscalculates a compression rate in compression coding of the image signalby a predetermined coding system based on the detection result from thefirst detection means and a detection result from the second detectionmeans.

According to the image compression processing device of the inventionset forth in claim 2, the second detection means detects characteristicsof horizontal and vertical low-frequency components of the image formedby the image signal. The calculation means calculates the compressionrate of the image signal also in consideration of the detection resultby the second detection means in addition to the detection result by thefirst detection means. Furthermore, in accordance with the compressionrate from the calculation means, the compression coding means cancompression-code the image signal into image data having a desiredcompression rate through one time of compression coding processing.

Due to this configuration, the compression rate of the image signal canbe calculated with higher accuracy also in consideration of thehorizontal low-frequency component and vertical high-frequency componentas the detection result by the second detection means, in addition tothe horizontal and vertical high-frequency components as the detectionresult by the first detection means.

An image compression processing device according to the invention setforth in claim 3 is dependent upon the image compression processingdevice set forth in claim 2, and further includes:

determination means that determines whether or not to adjust theinformation amount of the image signal to be stored in the storagemedium based on the detection result from the first detection means andthe detection result from the second detection means; and

information amount adjustment means that is provided upstream of therecording control means and adjusts the information amount of the imagesignal to be supplied to the recording control means if thedetermination means has determined that the information amount is to beadjusted.

According to the image compression processing device of the inventionset forth in claim 3, the determination means determines whether or notto adjust the information amount (data amount) of an image signal to beprocessed, based on horizontal and vertical high-frequency components ofthe image formed by the image signal to be processed, detected by thefirst detection means, and horizontal and vertical low-frequencycomponents of the image formed by the image signal to be processed,detected by the second detection means.

Specifically, if the determination means has determined that the imageincludes more low-frequency components than high-frequency componentsand hence involves small changes, the determination means determinesthat the information amount of the image signal forming the image is tobe adjusted. In this adjustment, the data amount is adjusted toward areduced amount. If it has been determined that the information amount ofthe image signal is to be adjusted, the information amount adjustmentmeans provided upstream of the recording control means adjusts theinformation amount of the image signal to be processed, followed byrecording of the resultant image signal in the storage medium via therecording control means.

This configuration offers effective use of the storage capacity of thestorage medium in which yet-to-be compression-coded image signals arealso recorded, and thus allows a larger amount of yet-to-becompression-coded image signals to be recorded in the storage mediumrapidly. That is, image signals themselves yet to be compression-codedcan be stored in the storage medium efficiently, which allows effectiveuse of the storage medium. Furthermore, when the image signal of whichinformation amount has been adjusted is retrieved from the storagemedium, the original image signal yet to be adjusted is restored fromthe retrieved after-adjustment image signal. Thus, this restored imagesignal can be utilized in a usual manner, such as beingcompression-coded and recorded in another recording medium.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram for explaining a digital camera to which oneembodiment of the invention is applied.

FIG. 2 is a block diagram for explaining a high-frequency integratingcircuit in the digital camera shown in FIG. 1.

FIG. 3 is a block diagram for explaining a low-frequency integratingcircuit in the digital camera shown in FIG. 1.

FIG. 4 is a block diagram for explaining a code amount predictorrealized by a CPU shown in FIG. 1 and an image compressor provided in animage codec.

FIGS. 5A to 5D are diagrams for explaining the relationship between YUVdata that is image data to be recorded in a storage medium of arecording device and stream data formed through compression coding.

FIG. 6 is a diagram showing one example of a quantization table.

FIG. 7 is a flowchart for explaining processing of data writing to amemory device (storage medium) 50.

FIG. 8 is a flowchart for explaining processing of data reading from thememory device (storage medium) 50.

FIG. 9 is a block diagram for explaining one example of an existingimage compression processing device.

FIG. 10 is a diagram for explaining an example of an image including fewhorizontal high-frequency components and much vertical high-frequencycomponents.

BEST MODE FOR CARRYING OUT THE INVENTION

One embodiment of a device, method and program according to the presentinvention will be described in detail below with reference to thedrawings. In the following, explanations will be made mainly on examplesin which a device, method and program according to the invention areapplied to a digital camera (imaging device) that captures static images(still images).

[Configuration of Digital Camera]

FIG. 1 is a block diagram for explaining the configuration of a digitalcamera of this embodiment. As shown in FIG. 1, the digital camera ofthis embodiment includes, in rough classification, an imaging element10, a pre-processing circuit 20, and a camera DSP (Digital SignalProcessor) 30. Furthermore, the digital camera includes a memory device50 that is an externally coupled apparatus, an LCD/monitor 51, arecording medium 52, a CPU (Central Processing Unit) 61 that functionsas a controller, a RAM (Random Access Memory) 62, an EEPROM(Electrically Erasable and Programmable Read Only Memory) 63, and anoperation unit 60 that accepts an instruction input from a user.

Referring to FIG. 1, the imaging element 10 is a component for capturingimage information, such as a CCD (Charge Coupled Device) or CMOS(Complementary Metal-Oxide Semiconductor).

The pre-processing circuit 20 includes a CDS (Correlated DoubleSampling)/AGC (Automatic Gain Control)/ADC (Analog/Digital Converter)circuit 21, a timing generator 22, and a V driver 23. Image informationthat has been captured through the imaging element 10 and converted intoan electric signal is supplied to the CDS/AGC/ADC circuit 21 in thepre-processing circuit 20.

The CDS/AGC/ADC circuit 21 executes CDS processing for the suppliedimage information to keep a favorable S/N ratio thereof, and executesAGC processing to control the gain thereof. Furthermore, the CDS/AGC/ADCcircuit 21 executes A/D conversion to thereby form image data convertedinto a digital signal. The above-described processes are carried out bythe CDS/AGC/ADC circuit 21. The timing generator 22 and the V driver 23control the drive timing of the imaging element 10 based on informationfrom the CDS/AGC/ADC circuit 21. The image data from the pre-processingcircuit 20 is supplied to the camera DSP 30.

As shown in FIG. 1, the camera DSP 30 includes a camera signalprocessing circuit 31, a high-frequency integrating circuit 32, alow-frequency integrating circuit 33, a to-be-recorded image producingcircuit 34, a resolution converting circuit 35, an image codec 36, adifferential signal processing circuit 37, a memory control circuit 38,a display control circuit 39, a medium control circuit 40, and a BIU(Bus Interface Unit) 41.

Furthermore, the memory device 50, the LCD (Liquid CrystalDisplay)/monitor 51, and the recording medium 52 are connected to thecamera DSP 30 as shown in FIG. 1. The memory device 50 is asemiconductor memory that has a comparatively large storage capacity,and is e.g. an SDRAM (Synchronous Dynamic Random Access memory). Therecording medium 52 is e.g. a so-called memory card employing asemiconductor memory. It should be obvious that it is also possible touse as the recording medium 52 any of optical recording media such asrecordable DVDs and recordable CDs, magnetic disks and other variousmedia.

The memory control circuit 38 controls writing of image data to thememory device 50 and reading-out of image data from the memory device50. The display control circuit 39 controls displaying of an image onthe LCD (Liquid Crystal Display)/monitor 51. The medium control circuit40 controls writing of image data to the recording medium 52 andreading-out of image data from the recording medium 52.

The BIU 41 is a block serving as an interface between the CPU 61 and thecamera DSP 30. Based on a signal from the operation unit 60 as a userinterface, the CPU 61 loads a program from the EEPROM 63 and executesthe program by use of the RAM 62 serving as a work memory, to therebyassure operation in accordance with a request from a user withcommunicating with the camera DSP 30.

The imaging operation by the digital camera of this embodiment iscarried out as follows. In response to an imaging operation instructionfrom a user accepted through the operation unit 60, the CPU 61 controlsthe imaging element 10 and the pre-processing circuit 20 so that theimage of a target object is captured as image data and the capturedimage data is supplied to the camera signal processing circuit 31, thehigh-frequency integrating circuit 32 and the low-frequency integratingcircuit 33 in the camera DSP 30.

The camera signal processing circuit 31 subjects the supplied image datato camera signal processing such as AF (Auto Focus), AE (Auto Exposure)and AWB (Auto White Balance), and supplies the processed image data tothe to-be-recorded image producing circuit 34. The to-be-recorded imageproducing circuit 34 produces YUV data that is image data to be recordedfrom the supplied image data that has been subjected to the camerasignal processing, and supplies the produced data to the differentialsignal processing circuit 37. In the present specification, the “YUVdata” as image data means image data composed of a Y signal (luminancesignal), a Cb signal (blue color difference signal), and a Cr signal(red color difference signal).

The differential signal processing circuit 37, for which a detaileddescription will be made later, implements differential compression forimage data (YUV data) to be recorded, to thereby suppress the dataamount of images recorded in the memory device 50 so that recording of alarger amount of image data therein is allowed. The image data processedby the differential signal processing circuit 37 is recorded in a memoryof the memory device 50 through the memory control circuit 38.

Thereafter, in accordance with control by the CPU 61, the memory controlcircuit 38 retrieves the newly recorded image data from the memorydevice 50, and supplies the retrieved data to the resolution convertingcircuit 35 through a system bus. The resolution converting circuit 35executes processing of changing the size of the image to be displayedbased on the supplied image data depending on the display screen of theLCD/monitor 51, and supplies the image data of which size has beenchanged to the display control circuit 39.

The display control circuit 39 creates an image signal (video signal) tobe supplied to the LCD/monitor 51 based on the image data from theresolution converting circuit 35, and supplies the created signal to theLCD/monitor 51. Accordingly, the image corresponding to the image datacaptured through the imaging element 10 is displayed on the displayscreen of the LCD/monitor 51, which allows a user to view the image.

Furthermore, if e.g. a shutter button provided in the operation unit 60is pushed down, the image data retrieved from the memory device 50 issupplied also to the image codec 36. The image codec 36 compresses thesupplied image data into stream data by a predetermined compressionsystem such as the JPEG system or MPEG system. Specifically, the imagecodec 36 executes time-to-frequency axis transform processing by the DCTor the like, quantization processing, and variable-length codingprocessing to thereby compress the image data.

The image data compressed by the image codec 36 is recorded in thememory device 50 via the differential signal processing circuit 37 andthe memory control circuit 38, or the image data compressed by the imagecodec 36 is recorded in the recording medium 52 via the medium controlcircuit 40.

In addition, if the CPU 61 is instructed to reproduce the image datarecorded in the recording medium 52 through the operation unit 60, theCPU 61 controls the medium control circuit 40 so that the target imagedata is retrieved from the recording medium 52 and is supplied to theimage codec 36. When compression-coded image data is supplied to theimage codec 36, the image codec 36 subjects the data to decodingprocessing to thereby restore the before-coding original image data, andsupplies the decoded data to the LCD/monitor 51 via the resolutionconverting circuit 35 and the display control circuit 39 so that theimage corresponding to the image data recorded in the recording medium52 is reproduced.

In the above-described manner, this digital camera captures the image ofa target object as image data into the memory device 50, and displaysthe image corresponding to the captured image data on the display screenof the LCD/monitor 51 so that the image can be viewed. Furthermore, if ashutter button in the operation unit 60 is pushed down, the digitalcamera executes data compression for the image data captured into thememory device 50 by use of a compression system such as the JPEG orMPEG, and allows the compressed data to be recorded in the memory device50 and the recording medium 52. In addition, the digital cameraretrieves the compressed image data recorded in the recording medium 52and decodes it, so that the image corresponding to the decoded imagedata can be displayed on the display screen of the LCD/monitor 51.

Moreover, the digital camera of this embodiment employs thehigh-frequency integrating circuit 32 and the low-frequency integratingcircuit 33 to thereby enable the image codec 36 to properly completeimage data compression processing (coding processing) through one timeof processing. Furthermore, in the digital camera, the differentialsignal processing circuit 37 executes differential signal processing,which allows more efficient data compression for yet-to-becompression-coded image data (YUV data) to be recorded in the memorydevice 50. Thus, the image data can be recorded in the memory device 50rapidly and the memory device 50 itself can be utilized efficiently.

[Configuration and Operation of High-Frequency Component IntegratingCircuit 32]

The high-frequency component integrating circuit 32 shown in FIG. 1 willbe described in detail below. As shown in FIG. 1, in the digital cameraof this embodiment, the high-frequency component integrating circuit 32provided in the camera DSP 30 accurately predicts the code amount ofdata compressed by a predetermined compression system such as the JPEGsystem or MPEG system. Furthermore, the high-frequency componentintegrating circuit 32 detects the characteristics of the horizontal andvertical directions of an image to be recorded in order to allow thedifferential signal processing circuit 37 to execute differential signalprocessing efficiently.

FIG. 2 is a block diagram for explaining the high-frequency componentintegrating circuit 32 used in the digital camera of this embodiment. Asshown in FIG. 2, the high-frequency component integrating circuit 32includes a luminance signal producer 321 and two processing systems forthe horizontal and vertical directions, respectively, of aprocessing-target image.

The processing system for the horizontal direction includes a horizontalhigh-frequency component extractor 322, an absolute value unit 323, anintegrator 324, and an area calculator 325. The processing system forthe vertical direction includes a vertical high-frequency componentextractor 326, an absolute value unit 327, an integrator 328, and anarea calculator 329.

The vertical high-frequency component extractor 326 includes ahorizontal band limiter 326 a, a horizontal interpolating/thinning unit326 b, a line memory 326 c, and a vertical arithmetic unit 326 d asshown in FIG. 2, to thereby allow the data amount of each line to bereduced to some extent so that sufficient processing is permitted evenif the storage capacity of the line memory is not large, as describedlater in detail.

Image data (captured image signal) from the pre-processing circuit 20 issupplied to the luminance signal producer 321. The luminance signalproducer 321 produces a luminance signal from the supplied image data.The luminance signal producer 321 may extract from the supplied imagedata, only pixel data that widely covers the frequency band of the image(e.g. green pixel data, in the case of a primary-color imager (image)),so that the extracted pixel data is used as a pseudo luminance signal.The luminance signal produced by the luminance signal producer 321 issupplied to the horizontal high-frequency component extractor 322 andthe vertical high-frequency component extractor 326.

If green pixel data (green color signal) is used as a pseudo luminancesignal, processing in the high-frequency component integrating circuit32 can be carried out accurately without a deficiency in data becausethe data amount has been already reduced.

Because image data is input in the order of horizontal scanning ingeneral, the horizontal high-frequency component extractor 322 can beformed of a high-pass filter having a comparatively simpleconfiguration. In contrast, the vertical high-frequency componentextractor 326 should form a high-pass filter for an image signal betweentwo or plural points having therebetween the time differencecorresponding to just one line. Therefore, in this embodiment, the linememory 326 c having a capacity equivalent to image data (captured imagesignal) corresponding to one line is provided for one line, or theplural line memories 326 c are provided for several lines.

However, provision of a line memory with a large storage capacity leadsto increases in the circuit scale and power consumption. To address thisproblem, the vertical high-frequency component extractor 326 in thisembodiment thins one-line image data to data of the necessary minimumnumber of pixels for the high-frequency component extractor 326,followed by storing of the resultant data in the line memory 326 c. Forthis purpose, the high-frequency component extractor 326 includes thehorizontal band limiter 326 a and the horizontal interpolating/thinningunit 326 b also as described above.

Specifically, in the vertical high-frequency component extractor 326,initially the horizontal band limiter 326 a limits a luminance signal(image signal) from the luminance signal producer 321 to onlypredetermined low-frequency components, to thereby prevent so-calledfold-back due to horizontal thinning processing. That is, the dataamount of the luminance signal is limited to a predetermined amount.

Subsequently, the horizontal interpolating/thinning unit 326 b executesinterpolation filter processing with use of multi-tap pixels in thehorizontal direction, and executes thinning processing for the luminancesignal so that the number of pixels of the resultant signal becomes anumber covered by the capacity of the line memory 326 c. The luminancesignal that has been thus subjected to pixel thinning is written to theline memory 326 c. As described above, in this embodiment, the verticalhigh-frequency component extractor 326 forms, with the minimum hardware,a high-pass filter that obtains a luminance signal between two or pluralpoints having therebetween the time difference corresponding to just oneline and calculates the difference between the points to thereby extracthigh-frequency components.

The high-frequency components of the luminance signal, obtained by thehorizontal and vertical high-frequency component extractors 322 and 326in the high-frequency component integrating circuit 32 shown in FIG. 2,are supplied to the corresponding absolute value units 323 and 327,respectively.

The absolute value unit 323 converts the horizontal high-frequencycomponents of the luminance signal from the horizontal high-frequencycomponent extractor 322 into the absolute values, in order to preventthe high-frequency components from canceling each other in theintegrator 324 at the subsequent stage. Similarly, the absolute valueunit 327 converts the vertical high-frequency components of theluminance signal from the vertical high-frequency component extractor326 into the absolute values, in order to prevent the high-frequencycomponents from canceling each other in the integrator 328 at thesubsequent stage.

Subsequently, the horizontal high-frequency components of the luminancesignal, converted into the absolute values by the absolute value unit323, are supplied to the integrator 324 and the area calculator 325. Thevertical high-frequency components of the luminance signal, convertedinto the absolute values by the absolute value unit 327, are supplied tothe integrator 328 and the area calculator 329.

The integrator 324 executes integration processing (accumulationprocessing) for the supplied horizontal high-frequency components of theluminance signal as the absolute values with defining e.g. one screen asthe processing unit region to thereby obtain horizontal high-frequencycomponent integration data, and supplies the data to the CPU 61 as acode amount predictor. Similarly, the integrator 328 executesintegration processing (accumulation processing) for the suppliedvertical high-frequency components of the luminance signal as theabsolute values with defining one screen as the processing unit regionto thereby obtain vertical high-frequency component integration data,and supplies the data to the CPU 61 as the code amount predictor.

It should be noted that the processing unit region of the integrationprocessing is not limited to one screen. The processing unit region ofthe integration processing may be an optionally specified region. It ispossible to use, as the optionally specified region, any of regionshaving various sizes, such as a macro block (region of 16×16 pixels), asub block (region of 8×8 pixels), a region equivalent to plural macroblocks, and a region equivalent to plural sub blocks.

The area calculator 325 counts the number of times of the integrationprocessing in the integrator 324 to thereby calculate the area of theregion including the horizontal high-frequency components of theluminance signal subjected to the integration processing, and suppliesthe calculation result to the CPU 61 as the code amount predictor.Similarly, the area calculator 329 counts the number of times of theintegration processing in the integrator 328 to thereby calculate thearea of the region including the vertical high-frequency components ofthe luminance signal subjected to the integration processing, andsupplies the calculation result to the CPU 61 as the code amountpredictor.

In the above-described manner, the following data are supplied from thehigh-frequency component integrating circuit 32 to the CPU 61 as thecode amount predictor: the integration data of the horizontalhigh-frequency components of the luminance signal (horizontalhigh-frequency component integration data); the data indicating the areaof the region including the horizontal high-frequency componentssubjected to the integration; the integration data of the verticalhigh-frequency components of the luminance signal (verticalhigh-frequency component integration data); and the data indicating thearea of the region including the vertical high-frequency componentssubjected to the integration.

The CPU 61 in the digital camera of this embodiment has also a functionas a code amount predictor also as described above. Furthermore, the CPU61 sets control data for the respective units in the high-frequencycomponent integrating circuit 32 shown in FIG. 2.

Specifically, the CPU 61 executes the following kinds of control:control of the kind and coefficient of the filter in the luminancesignal producer 321; control of high-luminance suppression processingand so on for a luminance signal; control of the kinds and coefficientsof the filters in the horizontal and vertical high-frequency componentextractors 322 and 326; control of the kind and coefficient of thefilter in the band limiter 326 a in the vertical high-frequencycomponent extractor 326; control of the interpolation filter andthinning rate of the horizontal interpolating/thinning unit 326 b;control of parameters of the absolute value units 323 and 327; andcontrol of region specification in the integrators 324 and 328.

[Configuration and Operation of Low-Frequency Component IntegratingCircuit 33]

The low-frequency component integrating circuit 33 shown in FIG. 1 willbe described below. The low-frequency component integrating circuit 33included in the digital camera of this embodiment has a configurationsimilar to that of the high-frequency component integrating circuit 32described above with FIG. 2, except that the low-frequency componentintegrating circuit 33 treats low-frequency components of a luminancesignal produced from image data.

FIG. 3 is a block diagram for explaining the low-frequency componentintegrating circuit 33 used in the digital camera of this embodiment. Asis apparent from a comparison between the low-frequency componentintegrating circuit 33 shown in FIG. 3 and the high-frequency componentintegrating circuit 32 shown in FIG. 2, the low-frequency componentintegrating circuit 33 shown in FIG. 3 has a configuration similar tothat of the high-frequency component integrating circuit 32 shown inFIG. 2, except that the low-frequency component integrating circuit 33includes a horizontal low-frequency component extractor 332 and avertical low-frequency component extractor 336 instead of the horizontalhigh-frequency component extractor 322 and the vertical high-frequencycomponent extractor 326 included in the high-frequency componentintegrating circuit 32 shown in FIG. 2.

The vertical low-frequency component extractor 336 includes a horizontalband limiter 336 a, a horizontal interpolating/thinning unit 336 b, aline memory 336 c, and a vertical arithmetic unit 336 d, similarly tothe vertical low-frequency component extractor 326 in the high-frequencycomponent integrating circuit 32 shown in FIG. 2. Processing-targetsignals for the horizontal low-frequency component extractor 332 and thevertical low-frequency component extractor 336 in the low-frequencycomponent integrating circuit 33 are low-frequency components in apredetermined band.

A luminance signal producer 331, absolute value units 333 and 337,integrators 334 and 338, and area calculators 335 and 339 in thelow-frequency component integrating circuit 33 are configured andoperate similarly to the corresponding units in the high-frequencycomponent integrating circuit 32 shown in FIG. 2, i.e., similarly to theluminance signal producer 321, the absolute value units 323 and 327, theintegrators 324 and 328, and the area calculators 3258 and 329,respectively.

Image data (captured image signal) from the pre-processing circuit 20 issupplied to the luminance signal producer 331 in the low-frequencycomponent integrating circuit 33 shown in FIG. 3. The luminance signalproducer 331 produces a luminance signal from the supplied image data.The luminance signal producer 331 also may extract from the suppliedimage data, only pixel data that widely covers the frequency band of theimage (e.g. green pixel data, in the case of a primary-color imager(image)), so that the extracted pixel data is used as a pseudo luminancesignal. The luminance signal produced by the luminance signal producer331 is supplied to the horizontal low-frequency component extractor 332and the vertical high-frequency component extractor 336.

Because image data is input in the order of horizontal scanning ingeneral also as described above, the horizontal low-frequency componentextractor 322 can be formed of a high-frequency trap filter or low-passfilter having a comparatively simple configuration. In contrast, thevertical low-frequency component extractor 336 thins one-line image datato data of the necessary minimum number of pixels for the low-frequencycomponent extractor 336, followed by storing of the resultant data inthe line memory 336 c. For this purpose, the low-frequency componentextractor 336 includes the horizontal band limiter 336 a and thehorizontal interpolating/thinning unit 336 b also as described above.

In this manner, in this embodiment, the vertical low-frequency componentextractor 336 forms, with the minimum hardware, a high-pass filter thatobtains a luminance signal between two or plural points havingtherebetween the time difference corresponding to just one line andcalculates the difference between the points. Furthermore, if the levelof the differential signal is equal to or lower than a predeterminedthreshold value, the low-frequency component extractor 336 determinesthat the signal is a low-frequency component and integrates thedifferential signal, to thereby extract low-frequency components.

The low-frequency components of the luminance signal, obtained by thehorizontal and vertical low-frequency component extractors 332 and 326in the low-frequency component integrating circuit 33 shown in FIG. 3,are supplied to the corresponding absolute value units 333 and 337,respectively, followed by being converted into the absolute valuestherein. Subsequently, the horizontal low-frequency components of theluminance signal, converted into the absolute values by the absolutevalue unit 333, are supplied to the integrator 334 and the areacalculator 335. The vertical low-frequency components of the luminancesignal, converted into the absolute values by the absolute value unit337, are supplied to the integrator 338 and the area calculator 339.

The integrator 334 executes integration processing for the suppliedhorizontal low-frequency components of the luminance signal as theabsolute values with defining e.g. one screen as the processing unitregion to thereby obtain horizontal low-frequency component integrationdata, and supplies the data to the CPU 61 as the code amount predictor.Similarly, the integrator 338 executes integration processing for thesupplied vertical low-frequency components of the luminance signal asthe absolute values with defining one screen as the processing unitregion to thereby obtain vertical high-frequency component integrationdata, and supplies the data to the CPU 61 as the code amount predictor.

The processing unit region of the integration processing is not limitedto one screen. Similarly to in the high-frequency component integratingcircuit 32 described above with FIG. 2, it is also possible to employany of optionally specified regions such as a macro block (region of16×16 pixels), a sub block (region of 8×8 pixels), a region equivalentto plural macro blocks, and a region equivalent to plural sub blocks.

The area calculator 335 counts the number of times of the integrationprocessing in the integrator 334 to thereby calculate the area of theregion including the horizontal low-frequency components of theluminance signal subjected to the integration processing, and suppliesthe calculation result to the CPU 61 as the code amount predictor.Similarly, the area calculator 339 counts the number of times of theintegration processing in the integrator 338 to thereby calculate thearea of the region including the vertical low-frequency components ofthe luminance signal subjected to the integration processing, andsupplies the calculation result to the CPU 61 as the code amountpredictor.

In this manner, the following data are supplied from the low-frequencycomponent integrating circuit 33 to the CPU 61 as the code amountpredictor: the integration data of the horizontal low-frequencycomponents of the luminance signal (horizontal low-frequency componentintegration data); the data indicating the area of the region includingthe horizontal high-frequency components subjected to the integration;the integration data of the vertical low-frequency components of theluminance signal (vertical low-frequency component integration data);and the data indicating the area of the region including the verticallow-frequency components subjected to the integration.

The CPU 61 in the digital camera of this embodiment has also a functionas a code amount predictor also as described above. Furthermore, the CPU61 sets control data for the respective units in the low-frequencycomponent integrating circuit 33 shown in FIG. 3.

Specifically, the CPU 61 executes the following kinds of control:control of the kind and coefficient of the filter in the luminancesignal producer 331; control of high-luminance suppression processingand so on for a luminance signal; control of the kinds and coefficientsof the filters in the horizontal and vertical high-frequency componentextractors 332 and 336; control of the kind and coefficient of thefilter in the band limiter in the vertical high-frequency componentextractor 336; control of the interpolation filter and thinning rate ofthe horizontal interpolating/thinning unit 336 b; control of parametersof the absolute value units 333 and 337; and control of regionspecification in the integrators 334 and 338.

[Configuration and Operation of Coding Predictor and Image Compressor]

A description will be made below on the CPU 61 as the code amountpredictor that is supplied with the horizontal high-frequencyintegration data and vertical high-frequency integration data from thehigh-frequency component integrating circuit 32 and the horizontallow-frequency integration data and vertical low-frequency integrationdata from the low-frequency component integrating circuit 33 asdescribed above, and executes code amount prediction based on thesedata. Furthermore, a description will be made on the image codec 36 asan image compressor that executes image compression processing. In thisembodiment, the CPU 61 realizes the function as the code amountpredictor by a program executed by the CPU 61 itself.

FIG. 4 is a block diagram for explaining the function of the CPU 61 asthe code amount predictor and the function of the image codec 36 as theimage compressor. As shown in FIG. 4, the CPU 61 as the code amountpredictor has the functions as a byte calculator 611, a quantizationscale calculator 612, and a quantization table creator 613. Furthermore,the image codec 36 as the image compressor has the functions as atime-to-frequency axis transformer (DCT unit) 361, a quantizer 362, anda variable-length coder 363.

As described with FIGS. 2 and 3, the horizontal and verticalhigh-frequency integration data from the high-frequency integratingcircuit 32 and the horizontal and vertical low-frequency integrationdata from the low-frequency integrating circuit 33 are supplied to thebyte calculator 611 realized by the CPU 61 as the code amount predictor.

Based on the high-frequency integration data and low-frequencyintegration data, the byte calculator 611 calculates the number of bytesof the compressed data of an image in the following manner.Specifically, if the image to be recorded is one that involves so-calleddrastic changes, i.e., includes large high-frequency components andsmall low-frequency components, the byte calculator 611 offers a largenumber as the number of bytes of the compressed data of the image. Incontrast, if the image is one that involves so-called small changes,i.e., includes small high-frequency components and large low-frequencycomponents, the byte calculator 611 offers a small number as the numberof bytes of the compressed data of the image. That is, the bytecalculator 611 properly grasps the number of bytes of theafter-compression data of a compression-target image to be recordeddepending on the degree of changes in the compression-target image.

The number of bytes of the after-compression image data, calculated bythe byte calculator 611, is supplied to the quantization scalecalculator 612. Based on the number of bytes of the after-compressionimage data calculated by the byte calculator 611, the quantization scalecalculator 612 calculates the quantization scales that allow therecording-target image to be compressed into a predetermined number ofbytes through only one time of processing, and supplies the calculatedquantization scales to the quantization table creator 613.

The quantization table creator 613 creates a quantization table based onthe quantization scales from the quantization scale calculator 612, andsupplies the quantization table to the quantizer 362 in the image codec36 as the image compressor.

In this manner, based on the image data of an image to be compressed,the CPU 61 as the code amount predictor accurately calculates the numberof bytes of the after-compression data dependent upon the complexity ofthe image to be compressed, based on the horizontal high-frequency andlow-frequency components and vertical high-frequency and low-frequencycomponents of the image.

Furthermore, based on the number of bytes of the after-compression dataof the compression-target image calculated by the byte calculator 611,the quantization scale calculator 612 calculates the quantization scalesfor compressing the compression-target image through one time ofcompression processing. Subsequently, a quantization table to be used inactual quantization processing is created by the quantization tablecreator based on the calculated quantization scales, followed by beingsupplied to the quantizer 362 in the image compressor 36. This operationsequence allows the image data of the compression-target image to becompression-coded with an optimal compression rate.

Meanwhile, image data read out from the memory device 50 is supplied viaa data bus to the time-to-frequency axis transformer 361 in the imagecodec, and is subjected to the discrete cosine transform (DCT) thereinso as to be transformed from time-axis components to frequency-axiscomponents, followed by being supplied to the quantizer 362. Thequantizer 362 adjusts the compression rate of the image data based onthe quantization table obtained from the quantization table creator 613.The variable-length coder 613 subjects the image data to variable-lengthcoding by use of variable-length codes such as the Huffman codes, andoutputs the coded data as final compressed image data.

As described above, the digital camera of this embodiment is greatlydifferent from the existing image compression device described with FIG.9, in that the digital camera of the embodiment achieves a largeadvantage that there is no need to repeat code amount predictionmultiple times with compressed image data being fed back but only onetime of compression processing can offer compressed image data. That is,the digital camera of the embodiment allows rapid image data compressionprocessing.

An actual operation mode is as follows. When a recording start button(or shutter button) in the operation unit 60 of FIG. 1 is operated by auser and thus the recording start timing is recognized (detected), thetiming is supplied to the CPU 61. The CPU 61 executes timing control forthe respective units so that the image data captured through the imagingelement at the moment of the recording start operation (or pushing downof a shutter) is subjected to the camera signal processing andto-be-recorded image production processing while the same image issubjected to the high-frequency component integration processing. Due tothis timing control, both the image on the path of the image processingand the image on the path of the high-frequency component integrationand code amount prediction can be matched to the original image, whichallows highly accurate code amount prediction.

In the digital camera of this embodiment, the byte calculator 611estimates the number of bytes of the after-compression image data(after-compression code amount) based on the output information from thehigh-frequency integrating circuit 32 and the low-frequency integratingcircuit 33. By thus comparing the output from the high-frequencyintegrator with that from the low-frequency integrator, theafter-compression code amount can be estimated with high accuracy.

If the accuracy of the estimation of the code amount of theafter-compression image data may be somewhat low, it is also possible,of course, for the byte calculator 611 to estimate the code amount ofthe after-compression image data by use of only the output from thehigh-frequency integrating circuit 32. Alternatively, the code amount ofthe after-compression image data may be calculated by use of not theoutput from the high-frequency integrating circuit 32 but only theoutput from the low-frequency integrating circuit 33.

[Explanation of Characteristic Part of Digital Camera]

In the digital camera of this embodiment shown in FIG. 1, thehigh-frequency integrating circuit 32 and the low-frequency integratingcircuit 33 correspond to features of the present invention. Therefore,with a focus on this point, a description will be made below oncharacteristic part of the digital camera shown in FIG. 1.

In the digital camera of this embodiment shown in FIG. 1, YUV image datathat has passed through the camera signal processing circuit 31 and theto-be-recorded image producer 34 and stream data arising from datacompression (compression coding) of the YUV image data in the imagecodec 36 are stored in e.g. a storage medium of the memory device 50.The YUV data and stream data have the data amount relationship likethose shown in FIGS. 5A to 5D.

The stream data shown in FIGS. 5A and 5B correspond to data resultingfrom data compression in the existing image compression device shown inFIG. 9 for example. The stream data shown in FIGS. 5C and 5D correspondto data resulting from data compression in the digital camera of thisembodiment. Furthermore, as the YUV data shown in these drawings, dataof which component magnitude is equivalent across all the spatialfrequencies is employed.

FIG. 5A shows the data amount relationship between the YUV data(yet-to-be compressed data) and stream data (after-compression data) ofan image in which the magnitudes of the components of all the spatialfrequencies are equal to each other. In FIG. 5A, the abscissa indicatesthe spatial frequency while the ordinate indicates the componentmagnitude. Because the component magnitude is equivalent across all thespatial frequencies, the YUV data is expressed by a flat waveform asindicated by the doted line in FIG. 5A.

In contrast, as for the stream data, the magnitudes of high-frequencycomponents thereof are small as indicated by the full line in FIG. 5A.This is because high-frequency components are compressed to a largerextent than low-frequency components by use of a compression codingsystem such as the JPEG system, by utilizing the facts that changes inimage information among adjacent pixels in a screen are moderate in manyof natural pictures, and that human's eyes are sensitive to moderateimage information changes but insensitive to sharp image informationchanges. Therefore, in the quantization processing by the image codec,high-frequency components resulting from the DCT are compressed moreintensely.

When data compression to obtain stream data like that indicated by thefull line in FIG. 5A is intended, if the existing image compressiondevice shown in FIG. 9 is used, it is needed that the values of thefixed-length quantization table 107 in the image compression deviceshown in FIG. 9 are set so that larger values are assigned tohigher-frequency components. FIG. 6 shows an example of values in thequantization table. The value at the upper left corner corresponds tothe DC component. A lower and righter value corresponds to a higherspatial frequency for both the horizontal and vertical directions.

The values of “99” in FIG. 6 correspond to the effect of compressinghigh-frequency components. The values in the table of FIG. 6 and thevalues output from the DCT unit 101 in the image compression deviceshown in FIG. 9 are combined with each other in the quantizer 102 inFIG. 9 for example, which offers small values as the values of theresultant stream data. Due to this operation, the value of “0 (zero)” isincluded in many high-frequency component data. The quantized data iscompressed by the variable-length coding (e.g., run-length coding) block103 in FIG. 9.

FIG. 5B shows the memory map on the memory device 50 in FIG. 1 for theYUV image data and stream data. The memory map of FIG. 5B results frommapping of the data amounts obtained by integrating the componentmagnitude in FIG. 5A with respect to the spatial frequency. FIG. 5Breveals that the stream data is smaller than the YUV data in size.

In the digital camera of the embodiment described above with FIG. 1, theoperation of estimating the code amount of after-compression image data,executed by the CPU 61 based on output information from thehigh-frequency integrating circuit 32 and the low-frequency integratingcircuit 33, is intended to determine the value of the spatial frequencyFth_1 at which the magnitude of the spatial frequency component ofstream data becomes small in FIG. 5A. The reason for the frequencydetermination is that the size of stream data recorded in the memorydevice 50 is finite.

FIG. 5C is a graph for explaining the relationship between YUV data andstream data mainly regarding the data amount thereof. The YUV data shownin FIG. 5C is obtained by compressing the YUV data shown in FIG. 5Athrough processing in the differential signal processing circuit 37 inthe digital camera of this embodiment shown in FIG. 1. The stream datashown in FIG. 5C results from data compression of the YUV data shown inFIG. 5A by a predetermined compression system such as the JPEG or MPEG.FIG. 5D is a diagram showing the state of the memory map on the memoryin which the YUV data and stream data shown in FIG. 5C have beenrecorded.

The stream data in FIG. 5C is the same as that shown in FIG. 5A. Incontrast, as for the YUV data, the component magnitude increases as thespatial frequency increases up to a special frequency Fth_2, and is keptat a constant value when the spatial frequency is higher than thefrequency Fth_2. This feature will be explained below.

A condition where an image includes much low spatial frequencycomponents means that the image can be compressed by use of differentialcompression such as DPCM (Differential Pulse Code Modulation). Forexample, in the case of monotonically increasing values such as 0, 1, 2,3, 4, the information thereof can be transmitted with only the startvalue (“0”, in this case) and the increase step (“1”) of the monotonicincreasing. If an image includes much low spatial frequency components,it is more advantageous to save the data of the image after subjectingthe data to the differential compression.

In the digital camera in FIG. 1, this differential compression isexecuted by the differential signal processing circuit 37. Specifically,in writing of YUV data to the memory device 50, the differential signalprocessing circuit 37 executes the differential compression for an imageincluding more low-frequency components than high-frequency components.On the other hand, when YUV data recorded in the memory device 50 afterthe differential compression is to be utilized, the differential signalprocessing circuit 37 executes differential expansion for the retrievedYUV data.

In FIG. 5C, the component magnitude of the YUV data is kept constantwhen the spatial frequency is the frequency Fth_2 or higher. This isbecause execution of the differential compression for components ofspatial frequencies equal to or higher than the frequency Fth_2 resultsin an increased data amount adversely, and therefore it is moreadvantageous to produce the data thereof without executing thedifferential compression.

For example, if differential compression is executed for data of whichcomponent magnitude toggles between 0 and 8 like 0, 8, 0, 8, 0, theresultant data is composed of the start value (“0”, in this case), theincrease step (+8), and the decrease step (−8). This differentialcompression is equivalent to conversion from the original data train 0,8, 0, 8, 0 to a data train 0, +8, −8, +8, −8. Accordingly, due to thebits for the signs, the data amount is increased as a result of thedifferential compression.

Therefore, in the digital camera of this embodiment, the differentialcompression is executed only when the data of a captured image includesmore low-frequency components than high-frequency components. Whether ornot to execute the differential compression may be determined on entirescreen basis. Alternatively, it is also possible to, for each section ofthe screen, compare the magnitude of low-frequency components with thatof high-frequency components and determine which way is employed for therecording in the memory.

If the data of a captured image includes both data subjected and notsubjected to the differential compression, identifiers fordifferentiating both the data are necessary. The identifiers may bestored in the setting register from the CPU 61 provided in thedifferential signal processing circuit 37 in FIG. 1 for use.Alternatively, the identifier may be added to YUV data of eachpredetermined unit so as to be recorded in the memory device 50 or thelike together with the YUV data.

In conclusion, any system is available as long as, when YUV datarecorded in the memory device 50 is retrieved for use, it can bedetermined accurately whether or not the retrieved YUV data has beensubjected to the differential compression and execution of differentialexpansion for only the YUV data subjected to the differentialcompression is assured.

As is apparent from a comparison between the memory map shown in FIG.5B, obtained when YUV data is recorded in the memory device 50 withoutbeing subjected to differential signal processing, and the memory mapshown in FIG. 5D, obtained when YUV data including more low-frequencycomponents than high-frequency components is recorded in the memorydevice 50 after being subjected to differential signal processing, usingthe differential signal processing can compress the YUV data itself andhence allows e.g. the entire one-screen YUV data to be stored and heldwith a smaller storage capacity.

Although the data amount of YUV data including much low-frequencycomponents can be reduced efficiently through differential signalprocessing, when the YUV data including much low-frequency components iscoding-compressed by a coding compression system such as the JPEG, thecompression rate is somewhat low, which results in stream data with alarge data amount.

In contrast, as for YUV data including much high-frequency components,differential signal processing is not executed because the data amountcannot be reduced efficiently. However, when YUV data including muchhigh-frequency components is coding-compressed by a coding compressionsystem such as the JPEG, the compression rate is high, which can offerstream data with a small data amount as shown in FIGS. 5A and 5B.

Therefore, if YUV data as yet-to-be compression-coded image data andstream data as compression-coded image data are recorded in the samestorage medium such as the memory device 50 like in the digital cameraof this embodiment, the data amounts of the YUV data and stream datachange depending on the frequency characteristic of the images asindicated by the doted line in FIG. 5D. Thus, the storage medium thatrecords both YUV data and stream data can be utilized adaptively andefficiently.

The unit of the differential signal processing may be one screen.However, in order to improve the effect, it is desirable to define asmaller unit region and execute the differential signal processing foreach unit region. The following configuration is available for example.Specifically, one screen is divided into a predetermined number of unitregions such as 8 regions or 16 regions. For each of the unit regions,horizontal and vertical high-frequency components and low-frequencycomponents are detected, so that whether or not to execute differentialsignal processing is determined depending on the detection result. It isalso possible, of course, to implement the detection and determinationfor each macro block or each sub block.

The differential signal processing may be executed so that original data(raw data not subjected to differential processing) is transferred onlyas the first data and differential data are transferred as all thesubsequent data. However, in order to eliminate influence of errors, itis desirable to transfer original data of a somewhat higher ratio todifferential data, such as to alternately transfer original anddifferential data, or to transfer one original data every time twodifferential data are transferred.

As described above, in the digital camera of this embodiment, based onthe characteristic of horizontal and vertical high-frequency components(state of high-frequency components) of a compression-target imageobtained by the high-frequency integrating circuit 32 and thecharacteristic of horizontal and vertical low-frequency components(state of low-frequency components) of the compression-target imageobtained by the low-frequency integrating circuit 33, the compressionrate of data compression by a predetermined compression coding systemsuch as the JPEG or MPEG is accurately predicted, which allowscompression coding to be properly carried out through one time ofcompression coding processing.

Furthermore, the differential signal processing circuit 37 is controlledbased on outputs from the high-frequency integrating circuit 32 and thelow-frequency integrating circuit 33 so that YUV data itself to berecorded in the memory device 50 is subjected to differentialcompression, which can achieve efficient use of the memory device 50.

[Data Writing/Reading to/from Memory Device 50]

Descriptions will be made below on processing of data writing to thememory device 50 and processing of data reading from the memory device50 executed in the digital camera of this embodiment shown in FIG. 1with reference to the flowcharts shown in FIGS. 7 and 8. FIG. 7 is aflowchart for explaining the processing of data writing to the memorydevice 50. FIG. 8 is a flowchart for explaining the processing of datareading from the memory device 50.

The processing of data writing to the memory device 50 will be describedbelow with reference to the flowchart of FIG. 7. Upon being instructedto write data to the memory device 50 through e.g. an instruction inputfrom a user, the CPU 61 starts the processing shown in the flowchart ofFIG. 7.

Initially, the CPU 61 reads out the registers of the high-frequencyintegrating circuit 32 and the low-frequency integrating circuit 33 inFIG. 1 via the BIU (Bus Interface Unit) for each part of an image (stepS101). Subsequently, for a supplied image signal, the CPU 61 determineswhether or not high-frequency components of the image are more dominantthan low-frequency components thereof (step S102). If it has beendetermined in the determination processing of the step S102 thathigh-frequency components are more dominant than low-frequencycomponents, the CPU 61 sets “0” in the register of the differentialsignal unit (step S103), and writes YUV data (image data) to therelevant region in the memory device 50 without executing differentialcompression therefor (step S104).

If it has been determined in the determination processing of the stepS102 that low-frequency components are more dominant than high-frequencycomponents, the CPU 61 sets “1” in the register of the differentialsignal unit (step S105), and writes YUV data (image data) to therelevant region in the memory device 50 after executing differentialcompression therefor (step S106).

Upon completion of the data writing processing in the step S104 or thewriting processing in the step S106, whether or not retrieval of alleffective images has been completed is determined (step S107). If it hasbeen determined that the retrieval has not been completed, theprocessing from the step S101 is repeated. If it has been determinedthat the retrieval has been completed, the processing shown in FIG. 7 isended.

The processing of data reading from the memory device 50 will bedescribed below with reference to the flowchart of FIG. 8. Theprocessing shown in FIG. 8 may be executed by the CPU 61 shown in FIG.1, or alternatively may be executed inside the differential signalprocessing circuit 37. In the following, an example in which theprocessing is executed inside the differential signal processing circuit37 will be described.

Upon receiving a data reading control signal due to control by the CPU61, the differential signal processing circuit 37 starts the processingshown in FIG. 8. Initially, the differential signal processing circuit 7reads out the value of a flag, which is its own register, indicatingwhether or not differential compression has been executed in datawriting regarding YUV data to be read out from the memory device 50(step S201). Subsequently, it is determined whether or not the value is“0”, which indicates that the data has not been subjected to thedifferential compression (step S202).

If it has been determined in the determination processing of the stepS202 that the value read out from the register is “0”, the YUV data tobe retrieved is data in the region corresponding to the non-execution ofthe differential compression. Therefore, the data is read out from thememory device 50 without being subjected to differential expansionprocessing (step S203). In contrast, if it has been determined in thedetermination processing of the step S202 that the value read out fromthe register is not “0”, the YUV data to be retrieved is data in theregion corresponding to the execution of the differential compression.Therefore, the data is read out from the memory device 50 with beingsubjected to differential expansion processing (step S204).

After the processing of the step S203 or the step S204, the registeraddress of the differential signal processing circuit 37 is incremented(step S205), followed by determination as to whether or not retrieval ofall effective images has been completed (step S206). If it has beendetermined in the determination processing of the step S206 that theretrieval of all effective images has not been completed, the processingis repeated from the S201. If it has been determined that the retrievalhas been completed, the processing shown in FIG. 8 is ended.

Thus, also for YUV data, which is yet-to-be compression-coded image datato be recorded in the memory device 50, the data amount thereof can beadjusted efficiently depending on the frequency characteristic of theimage formed by the YUV data. Furthermore, even if data is recorded inthe memory device 50 after the data amount thereof is adjusted, the datacan be properly restored when being retrieved, and used.

In the above-described embodiment, the memory control circuit 38realizes the function as the recording control means, the high-frequencyintegrating circuit 32 realizes the function as the first detectionmeans, and the low-frequency integrating circuit 33 realizes thefunction as the second detection means. Furthermore, a function of theCPU 61 as the code amount predictor realizes the function as thecalculation means that calculates a compression rate, and the imagecodec 36 realizes the function as the coding means.

In addition, the function as the information amount adjustment meansthat adjusts the information amount of an image signal is realized bythe differential signal processing circuit 37. The function as thedetermination means that determines whether or not to adjust aninformation amount is realized by the CPU 61 that receives a detectionoutput from the high-frequency integrating circuit 32 and a detectionoutput from the low-frequency integrating circuit 33.

It is also possible that e.g. the CPU 61 or a program (software)executed by the CPU 61 and the camera DSP realizes the functions of thefollowing units shown in FIG. 1: the high-frequency integrating circuit32, the low-frequency integrating circuit 33, the image codec 36, thedifferential signal processing circuit 37, the code amount predictor 61,and so on.

Although an example in which still image data is processed has beenexplained in the above description of the embodiment, the presentinvention is not limited to the example. Processing of moving image dataalso can apply the invention, almost similarly to the above-describedstill image data processing. That is, the invention can be applied alsoto a moving image recording function. Specifically, if code amountprediction is executed by use of high-frequency component integrationdata of a to-be-recorded moving image in parallel to signal processingfor the moving image, after-compression code amount or bit rate of themoving image can be controlled efficiently.

Although an example in which the invention is applied to a digitalcamera that mainly captures still images has been explained in the abovedescription of the embodiment, the present invention is not limited tothe example. The invention can be applied to the overall case ofexecuting compression processing for still image data and moving imagedata in any of the following various apparatuses: apparatuses that cancapture moving images such as so-called digital video cameras, cellphone terminals equipped with a camera function, PDAs equipped with acamera function, and information processing devices equipped with acamera function; DVD recording and reproducing apparatuses; CD recordingand reproducing apparatuses; recording and reproducing apparatusesemploying a hard disk as a recording medium; VTRs (Video TapeRecorders); personal computers; and other apparatuses.

INDUSTRIAL APPLICABILITY

The invention can avoid execution of multiple times of code amountprediction processing and image compression processing for acompression-target image, and hence can realize shortening of the periodfor the image compression processing. In addition, each of the codeamount prediction processing and image compression processing for thecompression-target image can be completed through one time ofprocessing, which can eliminate the need to provide a memory for storingtherein the entire data of a one-screen original image to be compressed.Therefore, an image compression processing device can be formed withoutincreases in the circuit scale and costs.

Furthermore, although compression processing is not repeated multipletimes for the same image, the accuracy of code amount prediction can beimproved and therefore compression processing for an image can beexecuted with a more proper compression rate because the code amount ofthe image is predicted by use of high-frequency components of the image.Accordingly, image quality deterioration due to the image compressioncan be reduced.

In addition, in processing of extracting high-frequency components of acompression-target image in order to predict the code amount of thecompression-target image, a thumbnail (preparatory image) of thecompression-target image or the like is not used but high-frequencycomponents are extracted from the compression-target image itself,followed by the code amount prediction based on the extractedcomponents. Therefore, enhanced accuracy of the code amount predictioncan be achieved.

Moreover, because code amount prediction is executed by using bothhorizontal and vertical high-frequency components, the accuracy of thecode amount prediction can be improved for an image involving imbalanceof the frequency band between the horizontal and vertical directions.That is, for compression-target images having various characteristics,the code amount prediction can be executed properly and hence thecompression rate can be defined accurately.

Furthermore, because horizontal pixel thinning processing is alsoemployed, even an extremely small capacity is enough as the capacity ofa memory for one line or several lines necessary to extract verticalhigh-frequency components of an image to be compressed. This feature canprevent an increase in the circuit scale and can reduce powerconsumption.

In addition, by implementing timing control so that an image to becompressed is matched to the image used for code amount prediction, theaccuracy of the code amount prediction for the image to be compressedcan be enhanced.

When the invention is applied to a camera (imaging device), thedistribution state of spatial frequencies included in an image to becompressed is determined, and the sizes of the image data (YUV data)subjected to camera signal processing and the after-compression data aredecided, to thereby reduce the data amount of the data to be written tothe memory. Thus, reduction in the memory capacity and power consumptioncan be realized. This feature allows the imaging device to realizeshortening of the imaging operation interval and hence repeatedlycapture images with short time intervals.

Furthermore, when the invention is applied to a camera (imaging device),if an image to be compressed includes much low-frequency components,image data (YUV data) subjected to camera signal processing is writtento the memory device after being subjected to differential compression.If the image does not include much low-frequency components, the data iswritten to the memory device without being subjected to the differentialcompression. Such control can optimize the memory capacity.

In addition, a determination is made by use of frequency integratingcircuits as to which of high-frequency components and low-frequencycomponents are more dominant as the characteristic of an image to becompressed. This feature can offer various advantages, such as one thatthe characteristic of the image can be determined properly and henceimage compression processing can be executed accurately and surely, andone that the memory capacity can be optimized.

1. An image compression processing device comprising: recording controlmeans that is supplied with an image signal and records the image signalin a storage medium; first detection means that is supplied with theimage signal and detects characteristics of a horizontal high-frequencycomponent and a vertical high-frequency component of the image signal;second detection means that is supplied with the image signal anddetects characteristics of a horizontal low-frequency component and avertical low-frequency component of the image signal; calculation meansthat calculates a compression rate in compression coding of the imagesignal by a predetermined coding system based on a detection result fromthe first detection means and a detection result from the seconddetection means; and coding means that executes compression coding forthe image signal or an image signal read out from the storage mediumbased on the compression rate calculated by the calculation means. 2.The image compression processing device according to claim 1, furthercomprising: determination means that determines whether or not to adjustan information amount of the image signal to be stored in the storagemedium based on the detection result from the first detection means andthe detection result from the second detection means; and informationamount adjustment means that is provided upstream of the recordingcontrol means and adjusts the information amount of the image signal tobe supplied to the recording control means if the determination meanshas determined that the information amount is to be adjusted.
 3. Animage compression processing device comprising: recording control meansthat is supplied with an image signal and records the image signal in astorage medium; first detection means that is supplied with the imagesignal and detects characteristics of a horizontal high-frequencycomponent and a vertical high-frequency component of the image signal;second detection means that is supplied with the image signal anddetects characteristics of a horizontal low-frequency component and avertical low-frequency component of the image signal; determinationmeans that determines whether or not to adjust an information amount ofthe image signal to be stored in the storage medium based on a detectionresult from the first detection means and a detection result from thesecond detection means; and information amount adjustment means that isprovided upstream of the recording control means and adjusts theinformation amount of the image signal to be supplied to the recordingcontrol means if the determination means has determined that theinformation amount is to be adjusted.
 4. An image compression processingmethod comprising: implementing control so that an image signal isrecorded in a storage medium; a first detection step for detectingcharacteristics of a horizontal high-frequency component and a verticalhigh-frequency component of the image signal; a second detection stepfor detecting characteristics of a horizontal low-frequency componentand a vertical low-frequency component of the image signal calculating acompression rate in compression coding of the image signal by apredetermined coding system based on a detection result in the firstdetection step and a detection result in the second detection step; andexecuting compression coding for the image signal or an image signalread out from the storage medium based on the compression ratecalculated in the calculation step.
 5. The image compression processingmethod according to claim 4, wherein calculating the compression rateincludes: calculating the number of bytes of after-compression data ofthe image signal based on the detection result in the first detectionstep or based on the detection result in the first detection step andthe detection result in the second detection step; calculating aquantization scale for compressing the image signal to a predeterminednumber of bytes through one time of compression coding processing basedon the calculated number of bytes of the after-compression data;creating a quantization table to be used for compression of the recordimage based on the calculated quantization scale; and indicating thecompression rate of the image signal by the created quantization table.6. The image compression processing method according to claim 4, whereinthe first detection step for detecting characteristics includes:producing from the image signal a luminance signal or a pseudo luminancesignal that includes a low-frequency component and a high-frequencycomponent dependent upon the image signal and has a frequencycharacteristic equivalent to a frequency characteristic of the luminancesignal; detecting the characteristic of the horizontal high-frequencycomponent of the image signal from the luminance signal or the pseudoluminance signal; and detecting the characteristic of the verticalhigh-frequency component of the image signal from the luminance signalor the pseudo luminance signal.
 7. The image compression processingmethod according to claim 6, wherein detecting the characteristic of thehorizontal high-frequency component includes: extracting the horizontalhigh-frequency component from the luminance signal or the pseudoluminance signal; converting the extracted horizontal high-frequencycomponent into an absolute value; and detecting the characteristic ofthe horizontal high-frequency component based on the horizontalhigh-frequency component converted into the absolute value, and thedetecting step of the characteristic of the vertical high-frequencycomponent includes the steps of: limiting a frequency band in order toadjust an information amount of the luminance signal or the pseudoluminance signal; executing horizontal thinning or interpolation of apixel for the luminance signal or the pseudo luminance signal for whichband limitation has been implemented; storing in a line memory theluminance signal or the pseudo luminance signal that has been subjectedto the thinning or interpolation and corresponds to one horizontal line;executing arithmetic operation between the luminance signal or thepseudo luminance signal that has been subjected to the thinning orinterpolation and corresponds to one horizontal line and the luminancesignal or the pseudo luminance signal delayed by one horizontal linefrom the line memory, to extract the vertical high-frequency component;converting the extracted horizontal high-frequency component into anabsolute value; and detecting the characteristic of the verticalhigh-frequency component based on the vertical high-frequency componentconverted into the absolute value.
 8. The image compression processingmethod according to claim 4, wherein detecting characteristics of ahorizontal low-frequency component and a vertical low-frequencycomponent of the image signal to be recorded in the storage mediumincludes: producing from the image signal a luminance signal or a pseudoluminance signal that includes a low-frequency component and ahigh-frequency component dependent upon the image signal and has afrequency characteristic equivalent to a frequency characteristic of theluminance signal; detecting the characteristic of the horizontallow-frequency component of the image signal from the luminance signal orthe pseudo luminance signal; and detecting the characteristic of thevertical low-frequency component of the image signal from the luminancesignal or the pseudo luminance signal.
 9. The image compressionprocessing method according to claim 8, wherein detecting thecharacteristic of the horizontal low-frequency component includes:extracting the horizontal low-frequency component from the luminancesignal or the pseudo luminance signal; converting the extractedhorizontal low-frequency component into an absolute value; and detectingthe characteristic of the horizontal low-frequency component based onthe horizontal low-frequency component converted into the absolutevalue, and detecting the characteristic of the vertical low-frequencycomponent includes: limiting a frequency band in order to adjust aninformation amount of the luminance signal or the pseudo luminancesignal, executing horizontal thinning or interpolation of a pixel forthe luminance signal or the pseudo luminance signal for which bandlimitation has been implemented, storing in a line memory the luminancesignal or the pseudo luminance signal that has been subjected to thethinning or interpolation and corresponds to one horizontal line,executing arithmetic operation between the luminance signal or thepseudo luminance signal that has been subjected to the thinning orinterpolation and corresponds to one horizontal line and the luminancesignal or the pseudo luminance signal delayed by one horizontal linefrom the line memory, to extract the vertical low-frequency component,converting the extracted horizontal low-frequency component into anabsolute value, and detecting the characteristic of the verticallow-frequency component based on the vertical low-frequency componentconverted into the absolute value.
 10. The image compression processingmethod according to claim 4, wherein whether or not to adjust aninformation amount of the image signal to be stored in the storagemedium is determined based on the detection result in the firstdetection step and the detection result in the second detection step,and if it has been determined that the information amount is to beadjusted, the information amount of the image signal to be recorded inthe storage medium is adjusted.
 11. The image compression processingmethod according to claim 10, wherein differential compressionprocessing for the image signal is executed in adjustment of theinformation amount of the image signal, and it is determined, based onthe characteristic of the horizontal high-frequency component, thecharacteristic of the vertical high-frequency component, thecharacteristic of the horizontal low-frequency component, and thecharacteristic of the vertical low-frequency component, that theinformation amount is to be adjusted if a low-frequency component ismore dominant than a high-frequency component in the image signal. 12.The image compression processing method according to claim 10 or claim11, wherein whether or not to adjust the information amount of the imagesignal is determined for each predetermined processing unit region of animage formed by the image signal.
 13. An image compression processingmethod comprising: recording an image signal in a storage medium; afirst detection step for detecting characteristics of a horizontalhigh-frequency component and a vertical high-frequency component of theimage signal; a second detection step for detecting characteristics of ahorizontal low-frequency component and a vertical low-frequencycomponent of the image signal; determining whether or not to adjust aninformation amount of the image signal to be stored in the storagemedium based on a detection result in the first detection step and adetection result in the second detection step; and an information amountadjustment step for adjusting the information amount of the image signalto be recorded in the storage medium if it has been determined in thedetermination step that the information amount of the image signal is tobe adjusted.
 14. The image compression processing method according toclaim 13, wherein differential compression processing for the imagesignal is executed in the information amount adjustment step, and indetermination whether or not to adjust an information amount, it isdetermined that the information amount is to be adjusted if alow-frequency component is more dominant than a high-frequency componentin the image signal, based on the detection result in the firstdetection step and the detection result in the second detection step.15. The image compression processing method according to claim 13 orclaim 14, wherein in determination determining whether or not to adjustan information amount, whether or not to adjust the information amountis determined for each predetermined processing unit region of an imageformed by the image signal.
 16. The image compression processing methodaccording to claim 15, wherein the method is used in an imaging deviceincluding imaging means that captures an image of a target object toform an image signal and outputs the image signal, and the image signalto be recorded in the storage medium is output from the imaging means.17. The image compression processing method according to claim 14,wherein the method is used in an imaging device including imaging meansthat captures an image of a target object to form an image signal andoutputs the image signal, and the image signal to be recorded in thestorage medium is output from the imaging means.
 18. The imagecompression processing method according to claim 13, wherein the methodis used in an imaging device including imaging means that captures animage of a target object to form an image signal and outputs the imagesignal, and the image signal to be recorded in the storage medium isoutput from the imaging means.
 19. An computer program productcomprising a tangible computer readable medium including program codethereon, the program code being executable to perform operationscomprising: implementing control so that an image signal is recorded ina storage medium; a first detection step for detecting characteristicsof a horizontal high-frequency component and a vertical high-frequencycomponent of the image signal; a second detection step for detectingcharacteristics of a horizontal low-frequency component and a verticallow-frequency component of the image signal; calculating a compressionrate in compression coding of the image signal by a predetermined codingsystem based on a detection result in the first detection step and adetection result in the second detection step; and executing compressioncoding for the image signal or an image signal read out from the storagemedium based on the compression rate calculated in the calculation step.20. A computer program product comprising a tangible computer readablemedium including program code thereon, the program code being executableto perform operations comprising: recording an image signal in a storagemedium; a first detection step for detecting characteristics of ahorizontal high-frequency component and a vertical high-frequencycomponent of the image signal; a second detection step for detectingcharacteristics of a horizontal low-frequency component and a verticallow-frequency component of the image signal; determining whether or notto adjust an information amount of the image signal to be stored in thestorage medium based on a detection result in the first detection stepand a detection result in the second detection step; and adjusting theinformation amount of the image signal to be recorded in the storagemedium if it has been determined in the determination step that theinformation amount of the image signal is to be adjusted.
 21. An imagecompression processing device comprising: a recording controller that issupplied with an image signal and records the image signal in a storagemedium; a first detector that is supplied with the image signal anddetects characteristics of a horizontal high-frequency component and avertical high-frequency component of the image signal; a second detectorthat is supplied with the image signal and detects characteristics of ahorizontal low-frequency component and a vertical low-frequencycomponent of the image signal; a calculator that calculates acompression rate in compression coding of the image signal by apredetermined coding system based on a detection result from the firstdetector and based on a detection result from the second detector; and acoder that executes compression coding for the image signal or an imagesignal read out from the storage medium based on the compression ratecalculated by the calculator.
 22. An image compression processing devicecomprising: a recording controller that is supplied with an image signaland records the image signal in a storage medium; a first detector thatis supplied with the image signal and detects characteristics of ahorizontal high-frequency component and a vertical high-frequencycomponent of the image signal; a second detector that is supplied withthe image signal and detects characteristics of a horizontallow-frequency component and a vertical low-frequency component of theimage signal; a determiner that determines whether or not to adjust aninformation amount of the image signal to be stored in the storagemedium based on a detection result from the first detector and adetection result from the second detector; and an information amountadjuster that is provided upstream of the recording controller andadjusts the information amount of the image signal to be supplied to therecording controller if the determiner has determined that theinformation amount is to be adjusted.